Python PETSc Example BP1 by tylercollins5737 · Pull Request #1806 · CEED/libCEED

tylercollins5737 · 2025-04-19T09:04:56Z

Hello!

This is a Python implementation of bpsraw.c for BP1 and was initially discussed in #1774 . I'm aiming to recreate the benchmark behavior using libceed and petsc4py. I appreciate any feedback or indications I'm headed in the right direction!

It does run, but this is a work in progress. Remaining tasks:

valeriabarra · 2025-05-20T16:34:25Z

@tylercollins5737 , thank you for your work. Are you able to address the reviewer's comments?

tylercollins5737 · 2025-05-20T22:54:10Z

@valeriabarra I was able to get it working using the "python" matrix type, MatAIJ was not the best. I believe the "python" type and MatCreateShell can achieve the same thing.

valeriabarra · 2025-10-08T21:08:57Z

Hey @tylercollins5737 , how are you? I was wondering if you intended to complete this PR or if there were any impeding issues that you encountered.

There are a couple of other potential contributors that are eyeing this project and I was wondering if there was a missing interface for which this BP could not be completed or if it was just for personal reasons. Thank you!

tylercollins5737 · 2025-10-09T22:45:19Z

Hey @tylercollins5737 , how are you? I was wondering if you intended to complete this PR or if there were any impeding issues that you encountered.

There are a couple of other potential contributors that are eyeing this project and I was wondering if there was a missing interface for which this BP could not be completed or if it was just for personal reasons. Thank you!

Hey @valeriabarra hope you are doing well! In short, progress didn't stop because of any major limitations in the libceed or petsc libraries. But the qfunction for building the RHS "SetupMassRHS" and "Error" I believe aren't available so a workaround would have to be implemented which could be very slow, or an addition to the gallery of qfunctions, or someway to use the qfunctions used by the BPs, or a way to define a custom one in python. If there are interested contributors they're more than welcome to work on it. If no one picks it up, I'll start on it again and get the development environment setup on a computer I actually own. Cheers!

jeremylt · 2025-10-10T14:45:35Z

Yeah, since there are QFunctions that are not available via the gallery, you'd have to use the technique shown in this folder for ex3_volume.py (and associated files) to use a QFunction written in C.

tylercollins5737 · 2025-10-10T16:39:26Z

@jeremylt Cool, yeah that should do the trick.

jeremylt · 2025-12-16T16:42:35Z

Hi @tylercollins5737, I'm hoping to have this ready in case next semester's students want to build on this for BP 2-6. This is good work so far and will be very helpful to future users. Is there anything I can do to help here? I'm happy to help out on any pieces that have you stuck so we can get this moving again.

tylercollins5737 · 2025-12-18T03:38:47Z

Hi @tylercollins5737, I'm hoping to have this ready in case next semester's students want to build on this for BP 2-6. This is good work so far and will be very helpful to future users. Is there anything I can do to help here? I'm happy to help out on any pieces that have you stuck so we can get this moving again.

Cool, thank you for the support. I have the development environment setup again with everything updated, and I'm making progress on the example! I will reach out if I get stumped.

zatkins-dev · 2026-07-08T21:26:47Z

@tylercollins5737 are you planning on coming back to this? I think there's a lot of good work here! If not, it may be time to close this PR.

tylercollins5737 · 2026-07-09T00:54:58Z

@tylercollins5737 are you planning on coming back to this? I think there's a lot of good work here! If not, it may be time to close this PR.

@zatkins-dev Yes! I should have have a pretty complete update soon, ideally by the end of the weekend.

zatkins-dev · 2026-07-09T02:29:38Z

Awesome! Happy to keep it open then, thanks!

tylercollins5737 · 2026-07-13T00:49:06Z

QFunctions are in the PR now (qfunctions/bps/bp1.h, common.h, plus qfunctions.c), using the ex3_volume.py approach you pointed me to. The gallery has Mass3DBuild/MassApply but nothing like SetupMassRhs or Error, so those have to be C QFunctions.
MatMult no longer copies data. The PETSc Vec arrays go to libCEED with CEED_USE_POINTER, on the host and on the device.
Added tests (BP1, degrees 1–3) and matched the CLI to bpsraw.c (-problem bp1, -ksp_max_it_clip, -write_solution).
Output matches bpsraw.c on CPU and GPU, across 1, 2 and 4 ranks and degrees 1–4. For example at 1 rank, degree 3: 16 iterations, max error 1.730043e-09, same as the C.

Two notes:

I copied bp1.h/common.h into examples/python/qfunctions/bps/, the way ex1–ex3 keep their own copies, so the example is self contained.
CeedVectorTakeArray isn't wrapped in the Python bindings, so the borrowed arrays get rebound on the next MatMult instead of detached. Works as written, but not the same as the C.

jeremylt · 2026-07-13T06:44:52Z

I have just a handful of comments, but this looks great!

tylercollins5737 · 2026-07-20T01:07:15Z

So the BP tests skip when petsc4py isn't installed, and right now that's every CI setup because GitHub Actions has no PETSc, and GitLab CI has PETSc but not petsc4py. So they only really run locally at the moment.

tylercollins5737 · 2026-07-26T20:07:33Z

assert: removed them, now matching ex_test.py.
names: renamed to test_bp1_degree_{1,2,3} — BP-keyed for the
BPs expansion.
take_array: thanks for python - add take_array #1995! bpsraw.py now detaches with
take_array, matching the C code.

jeremylt · 2026-07-27T08:49:50Z

Ok, we need a cross check from someone who has petsc-py set up locally, but this looks awesome and ready to go as far as I can tell

maybe @zatkins-dev has petsc-py?

zatkins-dev · 2026-07-27T14:02:36Z

I should be able to test locally today!

zatkins-dev · 2026-07-27T16:36:19Z

Looks like the source branch needs to be rebased/merged from main -- it still doesn't have the take_array binding.

zatkins-dev · 2026-07-27T16:39:07Z

Ok, we need a cross check from someone who has petsc-py set up locally, but this looks awesome and ready to go as far as I can tell

But we're passing once I do that rebase locally, so I think this is good to go.

zatkins-dev · 2026-07-27T16:39:46Z

@jeremylt we should be able to set up a test for this through CI too, I can draft an MR for it.

jeremylt · 2026-07-27T16:46:37Z

Yeah, would be good to add, I just didn't have time for it

The gallery has Mass3DBuild and MassApply, but nothing like SetupMassRhs or Error, which depend on the exact solution the benchmark solves for. These follow the ex1-ex3 pattern: C sources next to the Python examples, built into the QFunctions library by qfunctions.c.

Port examples/petsc/bpsraw.c to Python with petsc4py and the libCEED Python bindings, solving BP1. A python PETSc Mat takes the place of the MatShell; its mult() runs the libCEED operator. MatMult hands the PETSc Vec arrays straight to libCEED with CEED_USE_POINTER instead of copying them, on the host and on the device. petsc4py has no VecGetArrayAndMemType, so on the device the pointer comes from getCUDAHandle and is passed through the CUDA array interface.

The example picks its iteration limit from how long a first solve takes, so the tests pin it with ksp_max_it_clip to keep the iteration count the same every run, as the C test does.

ex1-ex3 only need libCEED, so moved the BP1 tests out of ex_test.py into their own bps_test.py, which skips itself when petsc4py is not available. The test target runs both.

Rename Split3, GlobalNodes, GlobalStart, CreateRestriction and ComputeErrorMax to snake_case, matching the rest of the file and ex1-ex3.

Gave every function, class and method a docstring with Args and Returns to match ex_common.py style.

Detached the arrays MatMult lends to libCEED with take_array from CEED#1995, matching CeedVectorTakeArray in the C.

tylercollins5737 · 2026-07-27T17:44:35Z

Looks like the source branch needs to be rebased/merged from main -- it still doesn't have the take_array binding.

Yeah my bad, I had just added those lines locally. Rebased onto main!

zatkins-dev · 2026-07-27T18:47:44Z

Looks good to me locally! I can add the test to CI in a follow-up.

zatkins-dev · 2026-07-27T18:52:29Z

Alternatively, @tylercollins5737 can add this line to .github/workflows/python-test-with-style.yml

diff --git a/.github/workflows/python-test-with-style.yml b/.github/workflows/python-test-with-style.yml
index 5cf592c7..38db3e27 100644
--- a/.github/workflows/python-test-with-style.yml
+++ b/.github/workflows/python-test-with-style.yml
@@ -27,6 +27,7 @@ jobs:
       run: |
         pip install -r requirements.txt
         pip install -r requirements-test.txt
+        pip install -r requirements-test-petsc.txt
     - name: Python test
       env:
         CC: ${{ matrix.compiler }}

and add a new file requirements-test-petsc.txt:

mpi4py
petsc
petsc4py

The python BP tests skip without petsc4py, so they were collected but never run in CI.

tylercollins5737 · 2026-07-27T22:35:08Z

Alternatively, @tylercollins5737 can add this line to .github/workflows/python-test-with-style.yml

diff --git a/.github/workflows/python-test-with-style.yml b/.github/workflows/python-test-with-style.yml
index 5cf592c7..38db3e27 100644
--- a/.github/workflows/python-test-with-style.yml
+++ b/.github/workflows/python-test-with-style.yml
@@ -27,6 +27,7 @@ jobs:
       run: |
         pip install -r requirements.txt
         pip install -r requirements-test.txt
+        pip install -r requirements-test-petsc.txt
     - name: Python test
       env:
         CC: ${{ matrix.compiler }}

and add a new file requirements-test-petsc.txt:

mpi4py
petsc
petsc4py

Cool, just pushed those changes.

zatkins-dev · 2026-07-27T22:46:09Z

let me try to diagnose, this is a PETSc issue

jeremylt reviewed Apr 21, 2025

View reviewed changes